Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let cts can skip commented cases in caselist file #430

Open
wants to merge 1 commit into
base: opengl-cts-4.6.0
Choose a base branch
from

Conversation

julizhan
Copy link

@julizhan julizhan commented Dec 5, 2023

Add a new else if in parseCaseList() to skip commented line of caselist file. With this we can skip some less important test cases that may cause system hang or crash and let the whole test finish.

Add a new else if in parseCaseList() to skip commented line of caselist
file. With this we can skip some less important test cases that may cause
system hang or crash and let the whole test finish.

Signed-off-by: Julia Zhang <[email protected]>
@CLAassistant
Copy link

CLAassistant commented Dec 5, 2023

CLA assistant check
All committers have signed the CLA.

@julizhan
Copy link
Author

@lordalcol @nico @ShabbyX Would you mind help to review this patch?

@ShabbyX
Copy link
Contributor

ShabbyX commented Aug 23, 2024

This probably works ok for a while, but wouldn't it be a hassle to keep the mustpass list in sync with upstream where you have locally / downstream commented some lines out?

May I suggest instead adding a step to your CI / runner that would:

  • Read the mustpass file
  • Read a file of suppressions / known failures / etc
  • Generate a new mustpass file with some lines removed based on the second file
  • Pass that new mustpass file to the CTS runner

The advantage of doing the above is that you have more control over how to suppress tests. For example, the suppression file could include globs (like test.prefix.*) and the generator could automatically remove all those tests without you having to comment them out one by one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants